As compared to conventional cathode-ray tubes (CRTs) primarily used for realizing moving images, LCDs (Liquid Crystal Displays) have a drawback, so-called motion blur, which is the blurring of outline of a moving portion perceived by a viewer when displaying a moving image. It is pointed out that this motion blur arises from the LCD display mode itself (see, e.g., Specification of Japanese Patent No. 3295437; “Ishiguro Hidekazu and Kurita Taiichiro, “Consideration on Motion Picture Quality of the Hold Type Display with an octuple-rate CRT”, IEICE Technical Report, Institute of Electronics, Information and Communication Engineers, EID96-4 (1996-06), p. 19-26″).
Since fluorescent material is scanned by an electron beam to cause emission of light for display in CRTs, the light emission of pixels is basically impulse-like although slight afterglow of the fluorescent material exists. This is called an impulse display mode. On the other hand, in the case of LCDs, an electric charge is accumulated by applying an electric field to liquid crystal and is retained at a relatively high rate until the next time the electric field is applied. Especially, in the case of the TFT mode, since a TFT switch is provided for each dot composing a pixel and each pixel normally has an auxiliary capacity, the ability to retain the accumulated charge is extremely high. Therefore, the light emission is continued until the pixels are rewritten by the application of the electric field based on the image information of the next frame or field (hereinafter, represented by the frame). This is called a hold display mode.
Since the impulse response of the image displaying light has a temporal spread in the above hold display mode, spatial frequency characteristics deteriorate along with temporal frequency characteristics, resulting in the motion blur. Since the human eye can smoothly follow a moving object, if the light emission time is long as in the case of the hold type, the movement of image seems jerky and unnatural due to the time integration effect.
To improve the motion blur in the above hold display mode, a frame rate (the number of frames) is converted by interpolating an image between frames in a known technology. This technology is called. FRC (Frame Rate Converter) and is put to practical use in liquid crystal displaying devices, etc.
Conventionally known methods of converting the frame rate include various techniques such as simply repeating read-out of the same frame for a plurality of times and frame interpolation using linear interpolation between frames (see, e.g., Yamauchi Tatsuro, “TV Standards Conversion”, Journal of the Institute of Television Engineers of Japan, Vol. 45, No. 12, pp. 1534-1543 (1991)). However, in the case of the frame interpolation processing using the linear interpolation, unnaturalness of motion (jerkiness, judder) is generated due to the frame rate conversion, and the motion blur disturbance due to the above hold display mode cannot sufficiently be improved, resulting in inadequate image quality.
To eliminate effects of the jerkiness, etc., and improve quality of moving images, a motion-compensated frame interpolation (motion compensation) processing using motion vectors is proposed. Since a moving image itself is captured to compensate the image movement in this motion compensation processing, highly natural moving images may be acquired without deteriorating the resolution and generating the jerkiness. Since interpolation image signals are generated with motion compensation, the motion blur disturbance due to the above hold display mode may sufficiently be improved.
Above Specification of Japanese Patent No. 3295437 discloses a technology of motion-adaptively generating interpolation frames to increase a frame frequency of a display image for improving deterioration of spatial frequency characteristics causing the motion blur. In this case, at least one interpolation image signal interpolated between frames of a display image is motion-adaptively created from the previous and subsequent frames, and the created interpolation image signals are interpolated between the frames and are sequentially displayed.
FIG. 1 is a block diagram of a schematic configuration of an FRC drive display circuit in a conventional liquid crystal displaying device and, in FIG. 1, the FRC drive display circuit includes an FRC portion 100 that converts the number of frames of the input image signal by interpolating the image signals to which the motion compensation processing has been given between frames of the input video signal, an active-matrix liquid crystal display panel 104 having a liquid crystal layer and an electrode for applying the scan signal and the data signal to the liquid crystal layer, and an electrode driving portion 103 for driving a scan electrode and a data electrode of the liquid crystal display panel 104 based on the image signal subjected to the frame rate conversion by the FRC portion 100.
The FRC portion 100 includes a motion vector detecting portion 101 that detects motion vector information from the input image signal and an interpolation frame generating portion 102 that generates interpolation frames based on the motion vector information acquired by the motion vector detecting portion 101.
In the above configuration, for example, the motion vector detecting portion 101 may obtain the motion vector information with the use of a block matching method and a gradient method described later or if the motion vector information is included in the input image signal in some form, this information may be utilized. For example, the image data compression-encoded with the use of the MPEG format includes motion vector information of a moving image calculated at the time of encoding, and this motion vector information may be acquired.
FIG. 2 is a diagram for explaining a frame rate conversion processing by the conventional FRC drive display circuit shown in FIG. 1. The FRC portion 100 generates interpolation frames (gray-colored images in FIG. 2) between frames with the motion compensation using the motion vector information output from the motion vector detecting portion 101 and sequentially outputs the generated interpolation signals along with the input frame signals to perform processing of converting the frame rate of the input image signal from 60 frames per second (60 Hz) to 120 frames per second (120 Hz).
FIG. 3 is a diagram for explaining an interpolation frame generation processing of the motion vector detecting portion 101 and the interpolation frame generating portion 102. The motion vector detecting portion 101 uses the gradient method to detect a motion vector 105 from, for example, a frame #1 and a frame #2 shown in FIG. 3. The motion vector detecting portion 101 obtains the motion vector 105 by measuring a direction and an amount of movement in 1/60 of a second between the frame #1 and the frame #2. The interpolation frame generating portion 102 then uses the obtained motion vector 105 to allocate an interpolation vector 106 between the frame #1 and the frame #2. An interpolation frame 107 is generated by moving an object (in this case, an automobile) from a position of the frame #1 to a position after 1/120 of a second based on the interpolation vector 106.
By performing the motion-compensated frame interpolation processing with the use of the motion vector information to increase a display frame frequency in this way, the display state of the LCD (the hold display mode) can be made closer to the display state of the CRT (the impulse display mode) and the image quality deterioration can be improved which is due to the motion blur generated when displaying a moving image.
In the motion-compensated frame interpolation processing, it is essential to detect the motion vectors for performing the motion compensation. For example, the block matching method and the gradient method are proposed as representative techniques for the motion vector detection. In these methods, the motion vector is detected for each pixel or small block between two consecutive frames and this motion vector is used to interpolate each pixel or small block of the interpolation frame between two frames. An image at an arbitrary position between two frames is interpolated at an accurately compensated position to convert the number of frames.
Since the frames are highly correlated in moving images and has continuity in the time axis direction, a pixel or a small block moving in one frame tends to move with the same movement amount in the subsequent frame or the previous frame. For example, in the case of a moving image of a ball rolling from right to left on a screen, the ball area moves with similar movement amounts in every frame. Consecutive frames tend to have the continuity of motion vectors.
Therefore, the motion vector in the next frame may more easily or more accurately be detected by reference to a motion vector detection result of preceding frames. For example, in the iterative gradient method, which is an improved gradient method, a motion vector of a neighboring block already detected in the previous frame or the current frame is defined as an initial deflection vector, which is used as a starting point to repeat calculations of the gradient method for a detected block. With this method, a substantially accurate movement amount can be acquired by repeating the gradient method about two times.
That is, in the iterative gradient method, as shown in FIG. 4(A), the sum of an initial deflection vector 110 of an already detected motion vector of a neighboring block 108 of a detected block, a first motion vector 111 acquired by a first gradient method, and a second motion vector 112 acquired by a second gradient method, is a motion vector 113 that is finally output.
Incidentally, when the FRC is considered to be realized by a real-time processing of a hardware or a simulation processing of a computer, etc., the calculation range for evaluating motion vectors needs to be limited in real use because of limitation to the circuit configuration or memory region of the hardware, or limitation to computer processing speed, etc.
For example, in an interpolation vector evaluating portion provided in the interpolation frame generating portion 102, the accuracy of the motion vector acquired by the calculations of the gradient method is evaluated by calculating the Displaced Field Difference (DFD) between the image information of the detected block and the image information of the block indicated by the motion vector from the detected block, and the vector evaluation calculation range needs to be limited since there is the restriction of the memory region saved for the image information in the evaluation region as well.
Note that, the DFD is an index that shows the degree of accuracy of a candidate vector, and the smaller a value of the DFD is, the better the matching is of a detected block and a block indicated by a motion vector from the detected block, and the more suitable a corresponding candidate vector is.
However, when the movement amount between frames is large, the vector acquired by the calculations of the gradient method may exceed the limited vector evaluation calculation range 109 in the above gradient method. That is, a vector is obtained by arithmetical calculations based on the gradient difference between the image information of previous and subsequent frames in the calculations of the gradient method, and therefore a vector exceeding the limited vector evaluation calculation range may be calculated.
When a vector exceeding the vector evaluation calculation range is calculated in this way in the motion vector calculation, the motion vector needs to be limited at an output stage of a motion vector detecting portion because of limitation to the memory for the image information in the following evaluation of an interpolation vector.
For example, FIG. 4(B) and FIG. 4(C) show the case where the second motion vector 112 acquired by the second gradient method exceeds the vector evaluation calculation range 109. In this case, there are considered various methods as to what kind of motion vector is finally output.
For an example thereof, as shown in FIG. 4(B), a vector acquired by the sum of the initial deflection vector 110, the first motion vector 111, and the second motion vector 112 is clipped at a maximum value within the vector evaluation calculation range 109 and output for the final motion vector 113.
Another method is that, as shown in FIG. 4(C), since the second motion vector 112 exceeds the vector evaluation calculation range 109, up to the first motion vector 111 are defined as effective vectors and the sum with the initial deflection vector 110 is output for the final motion vector 113.
In this manner, when a vector acquired by the calculations of the gradient method exceeds the vector evaluation calculation range, a specific processing is applied and a certain vector is output, however, the output vector is not faithfully reflecting the result of the calculations of the gradient method, etc., and is not an accurate motion vector. Therefore, when the motion-compensated frame interpolation processing is performed using the motion vector to which the specific processing has been given, a failure may occur in an interpolation image.
Even if the vector evaluation calculation range is set to be sufficiently enlarged, the specific processing is unnecessary, however, as the vector evaluation calculation range is enlarged, there is a higher possibility that an identical pattern (image information) with a detected block exists in other plurality of blocks within a screen or the calculation range and motion vector candidates increase, and therefore, it becomes difficult to detect an accurate motion vector.
Not only when using the iterative gradient method but also when using the block matching method as the method for detecting a motion vector, for example, the vector search range, etc., needs to be limited and it is difficult to output an accurate motion vector when the movement amount between frames is large as above.